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Abstract:  This  paper  presents  a  low-power  wireless  image 
sensor  node  for  military  surveillance,  with  a  noise-robust 
moving  object  detection  and  region-of-interest  based  rate 
controller.  The  improved  robustness  to  noise  from  both 
environment  and  hardware  further  reduces  the  transmission 
energy  with  negligible  computation  and  memory  overhead. 
The  rate  controller  minimizes  memory  requirement  by 
dynamically  tuning  the  coding  parameters  for  the  constant 
encoding  rate. 
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Introduction 

In  wireless  image  sensor  nodes  for  moving  object 
surveillance,  energy  efficiency  can  be  enhanced  through 
moving  object  detection  that  reduces  data  transmission  while 
preserving  the  information.  In  many  applications,  the  sensor 
nodes  are  exposed  to  noisy  environment  such  as  snow  or  rain, 
and  noise  can  also  be  induced  by  a  sensor  array  or  ADC 
during  image  sensing.  Even  under  such  noise,  reliable 
moving  object  detection  is  required  to  avoid  unnecessary 
transmission  of  background  scenes  [1].  Transmission  energy 
can  be  further  reduced  by  region-of-interest  (ROI)  based 
coding  that  allocates  more  bits  to  the  ROI  by  degrading  the 
non-ROI  quality  [2].  One  of  the  major  challenges  of  ROI 
coding  is  to  match  the  encoding  rate  with  the  available 
transmission  rate  under  the  variations  in  channel  bandwidth 
and  input  video  content,  since  the  mismatch  will  result  in 
random  drop  (quality  degradation)  or  buffering  (increase  in 
memory  requirement)  of  data. 

In  this  work,  we  present  a  low-power  wireless  image  sensor 
node  with  a  noise-robust  moving  object  detection  and 
region-of-interest  based  rate  controller  [Fig.  1].  The  moving 
object  detection  technique  is  improved  to  have  robustness 
noise  from  both  image  sensor  hardware  and  dynamic 
environment,  with  small  memory/computation  overhead. 
We  also  present  an  on-line  rate  controller  that  minimizes  the 
buffer  requirement  by  modulating  the  ROI  coding 
parameters  for  keeping  the  encoding  rate  under  the 
transmission  rate.  Simulation  results  show  that  the  system 
with  the  proposed  approach  consumes  significantly  lower 
energy  than  existing  complex  moving  object  detection 
methods  or  rate  controllers  at  the  same  ROI  quality. 
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Fig.  1.  Block  diagram  of  the  proposed  system.  Red  boxes 
indicate  newly-designed  blocks. 

First  Prototype  of  a  Self-Powered  Image  Sensor 

In  our  previous  work  [3],  we  have  presented  a  self-powered 
wireless  image  sensor  node  for  moving  object  surveillance 
[Fig.  2].  It  incorporates  the  energy  harvesting  image  sensor 
array  and  a  low-overhead  pre-processor  for  moving  object 
detection.  The  sensor  array  can  harvest  2.1pW  of  peak 
power  at  200klx  luminance,  which  can  self-power  the 
system  at  the  frame  rate  of  7  seconds  per  frame. 

In  this  paper,  we  focus  on  its  image  sensing  and  motion 
detection  performance.  As  Fig.  3(a)  shows,  the  image 
captured  by  the  sensor  node  has  high  level  of  random  and 
fixed-pattern  noise  with  very  limited  dynamic  range.  Even 
with  the  noisy  image  input,  the  pre-processor  performs  the 
expected  function  of  object  detection.  Fig.  3(b)  shows  that 
more  blocks  are  transmitted  in  frames  with  objects,  and  the 
number  of  transmitted  blocks  fairly  matches  with  the 
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Fig.  2.  (a)  Die  photo  and  (b)  key  performance  parameters 
of  the  first  prototype  of  our  image  sensor  node. 


Fig.  3.  (a)  Image  captured  by  the  sensor  node,  (b)  number 
of  transmitted  blocks  with  a  moving  object. 


simulation  result.  However,  due  to  the  random  noise  at  the 
background,  around  1/3  of  the  total  blocks  are  unnecessarily 
transmitted  even  when  there  is  no  moving  object.  Therefore, 
there  is  a  need  for  improving  the  noise  robustness  of  moving 
object  detection  to  further  reduce  the  transmission  energy. 
We  also  add  a  simple  rate  controller  with  a  low-overhead 
ROI  coding  method  to  further  optimize  the  memory 
requirement. 

Noise-robust  moving  object  detection 

Fixed-Pattern  Noise:  Fig.  4(b)  shows  an  image  with  FPN 
with  Gaussian  distribution  of  variation=0.05.  Although  FPN 
creates  vertical  lines  in  the  original  image  and  edge  map, 
their  locations  do  not  change  over  frames.  Therefore,  frame 
differencing  of  two  consecutive  edge  maps  removes  most  of 
the  lines  generated  by  FPN.  As  a  result,  increase  in  the 
activity  levels  at  the  background  due  to  FPN  is  not 
significant,  generating  similar  distribution  of  the  activity 
map  as  the  original  image.  Therefore,  it  can  be  claimed  that 
certain  level  of  FPN  can  be  effectively  handled  by  the  nature 
of  ED+FD  method. 

Random  Noise:  If  the  pixels  in  an  input  image  are  affected 
by  the  random  noise,  their  boundary  pixels  will  be 
determined  as  edges  in  an  edge  map.  As  the  location  of  noisy 
pixels  changes  frame  by  frame,  the  edge  locations  also 
change,  resulting  in  large  activity  level  at  the  background 
due  to  noise  [Fig.  4(c)].  We  propose  removing  the  edges 
generated  by  impulse  noise  in  each  edge  map,  instead  of 
filtering  the  original  image.  To  remove  impulse  edge  pixels 


Fig.  4  (left):  input  images,  (middle):  edge  map,  (right): 
block-level  activity  map  with  (a)  original  video,  (b)  with  FPN, 
(c)  with  random  noise  using  the  original  ED+FD  method,  (d) 
images  with  random  noise  using  the  proposed  noise-robust 
method. 
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Fig.  5.  Diagram  of  (a)  noise-robust  moving  object  detection 
and  (b)  rank  closing. 


in  the  edge  map,  we  use  a  simple  median  filter  with  3x3 
kernel  size.  Median  filtering  on  an  image  generally  requires 
sorting  elements  in  a  kernel,  which  needs  computation  of 
0(nlogn).  However,  as  we  perform  filtering  on  a  bit-map  of 
edges,  it  can  be  implemented  simply  by  adding  the  bits  in  the 
kernel  and  comparing  the  value  with  the  threshold  (5  for 
median  filter).  Fig.  4(d)  shows  the  edge  map  after  the 
proposed  filtering  process.  It  shows  that  most  of  the  small 
group  of  edges  generated  by  random  noise  are  removed, 
resulting  in  the  similar  activity  map  as  the  original  video. 

Noise  from  dynamic  environment :  The  noise  from  dynamic 
environment  (e.g.  snowflakes)  usually  cover  a  group  of 
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Fig.  6.  Diagram  of  the  proposed  non-ROI  bit  truncation 
method. 

pixels  (1-2  blocks),  and  cannot  be  eliminated  by  the  pixel- 
level  filtering.  Therefore,  we  propose  to  apply 
morphological  filtering  to  the  block-level  activity  map 
generated  by  edge  detection  and  frame  differencing  (ED+FD) 
[Fig.  5].  For  low-overhead  filtering,  we  utilize  rank  closing, 
a  series  of  simple  filtering  operations  with  high/low  order 
values.  With  the  rank  closing  operation,  noisy  blocks  with 
high  activity  levels  are  removed,  while  the  regions  with  low 
activity  levels  inside  the  object  are  filled. 


ROI-based  rate  control 


Once  the  ROIs  are  determined,  energy-efficiency  can  be 
further  improved  by  assigning  more  resources  to  ROI.  Since 
dropping  non-ROIs  leads  to  overall  quality  degradation  and 
loss  of  context,  we  consider  transmitting  non-ROIs  with 
smaller  size.  For  higher  compression  of  the  non-ROI  at 
MJPEG,  we  reduce  the  pixel  value  variance  in  non-ROI 
blocks  using  bit-truncation  [Fig  6].  Bit-truncation  also 
reduces  the  energy  dissipation  of  MJPEG.  To  match  the 
encoding  and  transmission  rates,  we  design  an  on-line  rate- 
controller  that  tunes  the  three  parameters  (the  ROI  detection 
threshold,  the  truncation  level  of  non-ROI  blocks,  and  the 
QF  of  MJPEG)  [Fig.  7].  Fig.  8  shows  that  the  rate  controller 
with  truncation  provides  higher  ROI  quality  under  a  given 
target  data  rate. 


System  Design  and  Energy/Performance  Analysis 

An  image  processing  engine  with  the  proposed  approaches 
is  synthesized  with  the  IBM  130nm  technology  [Fig.  9]  for 
energy/area  analysis. 

We  first  compare  performance  and  energy  consumption  of 
the  proposed  rate  controller  with  the  existing  complex  rate 
controllers  in  H.264/AVC.  Fig.  10  shows  that  the  proposed 
approach  requires  more  data  transmission  to  achieve  the 


Fig.  8.  Performance  and  quality  of  different  ROI  coding 
methods. 


same  quality.  However,  with  its  significantly  low 
computation  energy  and  reduced  buffer  overhead,  it 
consumes  less  system  energy  than  other  approaches  at  the 
same  ROI  quality. 

For  analysis  on  noise  robustness,  we  compare  the  detection 
performance  with  existing  complex  moving  object  detection 
methods  based  on  Optical  Flow  (OF)  [4]  and  Gaussian 
Mixture  Model  (GMM)  [5].  Fig.  11  shows  data  reduction 
and  system  energy  consumption  at  ROI  delivery=0.8  under 
various  types  of  noise. 

With  FPN,  the  moving  object  detection  method  based  on 
ED+FD  shows  comparable  noise-robustness  to  the  complex 
methods  such  as  OF  and  GMM.  However,  in  presence  of 
random  noise,  the  performance  of  the  original  ED+FD 
method  significantly  degrades  since  random  noise  increases 
activity  level  (movement  of  edges)  at  the  background. 
However,  after  performing  edge  map  filtering,  the  non-ROI 
activity  level  significantly  decreases,  thereby  reducing  false 
detection.  For  videos  with  environmental  noise  (snow),  the 
proposed  filtering  scheme  on  the  block-level  activity  map 
improves  detection  performance  of  ED+FD.  However,  the 
proposed  method  does  not  always  outperform  the  complex 
methods  in  detecting  ROI,  resulting  in  higher  transmission 
energy  in  some  cases. 

Although  OF  and  GMM  consumes  less  transmission  energy 
due  to  higher  data  reduction,  they  require  significant 
memory  size  (OF:  200  bit/pixel,  GMM:  120  bit/pixel).  As  a 
result,  the  leakage  power  of  the  SRAM  for  these  methods 
(OF:  84  uW,  GMM:  50  uW)  is  much  higher  than  the 
harvested  power  at  the  maximum  brightness  (2.1  uW). 
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Fig.  9.  (a)  Block  diagram,  (b)  layout,  and  (c)  area/energy  of  the  proposed  wireless  image  sensor  system. 
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Fig.  10.  (a)  Quality-energy  and  (b)  energy  @ROI 
quality=0.8  of  different  rate  controllers. 


Therefore,  the  system  with  those  high  memory  demand 
moving  object  methods  cannot  be  self-powered  by  energy 
harvesting.  A  low  memory  requirement  (storage:  1  bit/pixel, 
access:  2  bit/pixel)  of  the  proposed  moving  object  detection 
method  enables  the  self-supported  operation  with  a  wireless 
transmitter  at  frame  rate  of  95  seconds/frame  when  FPN 
exists.  In  case  of  random  and  environmental  noise,  the  self¬ 
power  performance  of  the  original  ED+FD  method 
significantly  degrades  because  of  the  large  transmission 
energy  consumption  due  to  the  false  detection.  By  reducing 
false  detection  by  noise  robustness,  the  proposed  approach 
can  be  self-powered  at  much  higher  frame  rate  than  ED+FD. 


Conclusion 

We  have  presented  an  energy-efficient  wireless  image 
sensor  with  noise-robust  motion  detection  and  ROI-based 
rate  controller.  Its  improved  robustness  to  both 
environmental  and  hardware-induced  noise  reduces  the 
transmission  energy  with  the  reliable  ROI  delivery.  The  rate 
controller  modulates  coding  parameters  to  keep  encoding 
rate  constant  under  variations  in  channel  and  input  video, 
thereby  reducing  the  buffer  size.  The  improved  noise-  and 
variation-robustness  will  enhance  reliability  and  energy- 
efficiency  of  the  system,  which  are  the  key  requirements  of 
the  military  applications. 
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Fig.  11.  (a)  Data  reduction  and  (b)  system  dynamic  energy 
consumption  through  different  moving  object  detection 
methods  under  noise. 
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